The Importance of Parameter Tuning in Text Categorization

Text Categorization algorithms have a large number of parameters that determine their behaviour, whose effect is not easily predicted objectively or intuitively and may very well depend on the corpus or on the document representation. Their values are usually taken over from previously published results. In this article we investigate the effect of parameter tuning on the accuracy of two Text Categorization algorithms: the well-known Rocchio algorithm and the lesser-known Winnow. It turns out that the optimal parameter values are sometimes very different from those found in literature. We show that parameter tuning can greatly improve the accuracy of both algorithms, much more so than Term Selection. We also show that a non-Euclidian variant of the Vector Space Model may improve the accuracy but that at optimal parameter values this effect practically disappears.